Method for image animation using image value rules

ABSTRACT

A method for image presentation are provided, in which at least one theme for image presentation is obtained; at least one set of images is acquired for presentation in association with each theme; and a presentation area image is generated for presentation on a display with a plurality of separate presentation objects in the presentation area image; an image value for each of the acquired images is determined; an emphasis score is determined for each of the acquired images. The presentation area image is presented with one of the acquired images in each of the presentation objects; and the presentation objects are animated by moving the presentation objects relative to each other in a manner that attracts more attention to presentation objects that are used to present images having a higher emphasis score than presentation objects that are used to simultaneously present images having a lower emphasis score.

FIELD OF THE INVENTION

The present invention relates to the production and viewing of animage-based presentation of digital media from a digital mediacollection.

BACKGROUND OF THE INVENTION

With the advent of inexpensive, portable digital imaging devices, therehas been a dramatic growth of digital images captured. In addition,since the capture of digital images no longer require the cost of filmand the corresponding printing of the film images, there are littleconstraints to the number of images that can be taken with digitaldevices. Multi-gigabyte camera cards and near terabyte computer harddrives enable the retention of digital image files more than everbefore. However, the viewing of these images still requires time tocomprehend the context and subject of digital images. A computer slideshow is able to present a mere fraction of a typical image collection inthe duration that a consumer is willing to sit and watch the images fadein and out on a screen. Yet even when playing a slide show, it isdifficult to view images that are not displayed in proper sequencing orin context with others.

Furthermore, organizing images for viewing is a time consuming task thatfew are willing to take on. Although tagging of images enables key wordsto be searched using basic search engines, there still remains a needfor the indexing of image content in conjunction with the playback andvisualization of these images in context from one's image collection. Inaddition, it is an unmet need for presenting images that are logicallyrelated with each other in a manner that emphasizes certain images overothers. Moreover, a large number of images should be cued and presentedsimultaneously in a rapid manner using an entertaining animation to takeadvantage of the peripheral vision of a viewer without compromising aviewer's comprehension of the visual content. It is the purpose of thisinvention to overcome these deficiencies over the current state of theart.

SUMMARY OF THE INVENTION

It is an object of the present invention to logically group and presentan increased amount of digital media on the same presentation area. Thisobject is achieved by a method of image-based presentation comprising:

(a) Obtaining at least one set of images for image presentation;

(b) Generating a presentation area image for presentation on a displayhaving a plurality of separate presentation objects in the presentationarea;

(c) Determining an image value for each of the acquired images byanalyzing the images according to an image value metric;

(d) Presenting the presentation area image with one of the acquiredimages in each of the presentation objects and with the presented imagesbeing selected such that there will be at least a range of differentimage values for the images presented at any one moment in time;

(e) Animating the presentation objects within the presentation areaimage by moving the presentation objects relative to each other in amanner that attracts more attention to presentation objects that areused to present images having a higher image value than presentationobjects that are used to simultaneously present images having a lowerimage value.

This method has the advantage of producing a presentation from a givenset of digital media that can be viewed by a person. It also enablesmany images to be selectively viewed within a given time sequence.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter of the invention is described with reference to theembodiments shown in the drawings.

FIG. 1 is a block diagram of a digital multimedia presentation andcapture device that can implement the present invention;

FIG. 2 is a database and software block diagram of an embodiment of thepresent invention;

FIG. 3 is a flow diagram showing a method of image based presentation;

FIG. 4 is a representation of a presentation area containingpresentation objects;

FIG. 5 is representation of layers of image sets and motion vectors;

FIG. 6A is a representation of a theme of presentation objects in amotion metaphor of panning along water and rocks in a stream;

FIG. 6B is a representation of a navigation theme;

FIG. 6C is a representation of a centered presentation object andcircling presentation objects;

FIG. 6D is a representation of a building theme;

FIG. 7A is a representation of Christmas themes;

FIG. 7B is a representation of party themes;

FIG. 7C is a representation of a flocking theme;

FIG. 7D is a representation of a passage theme;

FIG. 8A is a representation of media playing from remote media source;

FIG. 8B is a representation of a media library;

FIG. 9 is a representation of a theme of a menu of themes;

FIG. 10A is a representation of an interaction with moving presentationobjects;

FIG. 10B is a representation of the paused objects after interaction;and

FIG. 11 is a representation of segmented portions of images used forpresentation.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, some embodiments of the present inventionwill be described as software programs. Those skilled in the art willreadily recognize that the equivalent of such a method can also beconstructed as hardware or software within the scope of the invention.

Because image manipulation algorithms and systems are well known, thepresent description will be directed in particular to algorithms andsystems forming part of, or cooperating more directly with, the methodin accordance with the present invention. Other aspects of suchalgorithms and systems, and hardware or software for producing andotherwise processing the image signals involved therewith, notspecifically shown or described herein can be selected from suchsystems, algorithms, components, and elements known in the art. Giventhe description as set forth in the following specification, allsoftware implementation thereof is conventional and within the ordinaryskill in such arts.

FIG. 1 is a block diagram of one embodiment of a multimedia presentationdevice 301 taking the form of a digital camera phone. Other embodimentsof multimedia presentation device 301 can include televisions, portableor non-portable computers and displays, digital cameras or media playerssuch as those with internal memory or those with removable memory suchas a Digital Versatile Disk or gaming platforms. A multimediapresentation device 301 can be a portable battery operated device, smallenough to be easily handheld by a user when capturing and reviewingimages.

In the embodiment of FIG. 1, multimedia presentation device 301 includesa CMOS image sensor 311 having a lens system 305 that focuses light froma scene (not shown) onto an image sensor array 314. Lens system 305 canhave a single lens or it can have one or more elements. Lens system 305can be of a fixed focus type or can be manually or automaticallyadjustable. Lens system 305 is optionally adjustable to provide avariable zoom that can be varied manually or automatically. Other knownarrangements can be used for lens system 305.

Light from the scene that is focused by lens system 305 onto imagesensor array 314 is converted into image signals representing an imageof the scene. Image sensor array 314 can take any number of forms andcan, for example, comprise a charge couple device (CCD), a complimentarymetal oxide sensor (CMOS), or any other electronic image sensor known tothose of ordinary skill in the art. Image sensor array 314 provides anumber of discrete sensing locations known as picture elements or“pixels” forming the image. The exact number of pixels provided by imagesensor array 314 is not critical. In one non-limiting example, the imagesensor array can comprise an arrangement of 1280 columns×960 rows ofpixels.

In the embodiment of FIG. 1, image sensor array 314 comprises a CMOSimage sensor 311 that can provide color image information using thewell-known Bayer color filter pattern and is controlled by timinggenerator 312, which also controls an optional flash 303 in order toilluminate the scene when the ambient illumination is low.

In the embodiment of FIG. 1, image sensor array 314 provides analogsignals representing an amount of light incident on each pixel during anexposure period. The analog signals are amplified and converted todigital data by an analog-to-digital (A/D) converter circuit 316 which,in this embodiment is formed on CMOS image sensor 311. However, in otherembodiments, image sensor array 314 can provide digital data withoutconversion.

The digital data from image sensor array 314 is stored in a DRAM buffermemory 318 and subsequently processed by a digital processor 320 inaccordance with instructions in digital processor 320 or in accordancewith programs stored in firmware memory 328, which can be flash EPROMmemory. The digital processor 320 can include a real-time clock 324,which keeps the date and time even when the multimedia presentationdevice 301 and digital processor 320 are in their low power state.Digital processor 320 stores the processed digital data as a digitalimage in a memory 330.

The digital image can comprise one or more still images, multiple stillimages and/or a stream of apparently moving images such as a videosegment. Where the digital image comprises a stream of apparently movingimages, the digital image can comprise image data stored in aninterleaved or interlaced image form, a sequence of still images, and/orother forms known to those of skill in the art of digital video. In someembodiments, multimedia presentation device 301 can also store videoclips by summing multiple pixels of the image sensor array 314 together(e.g. summing pixels of the same color within each 4 column×4 row areaof the image sensor array 314) to produce a lower resolution video imageframe. The video image frames are read from the image sensor array 314at regular intervals, for example, using a 30 frame per second readoutrate.

Digital processor 320 can apply various image processing algorithms tothe image signals when forming a digital image. These can include, butare not limited to, color and exposure balancing, interpolation andcompression. Where the image signals are in the form of analog signals,digital processor 320 also converts these analog signals into a digitalform. For example, in one embodiment of the still image mode, digitalprocessor 320 performs color interpolation followed by color and tonecorrection, in order to produce rendered sRGB image data or other formsof rendered digital image data.

Digital processor 320 can also be used to generate metadata inassociation with each image. Metadata is data that is related to adigital image or a portion of a digital image but that is notnecessarily observable in the image data itself. In this regard, digitalprocessor 320 can receive signals from user controls 334, audio codec340 and, optionally, generates metadata based upon such signals. Themetadata can include, but is not limited to, information such as thetime, date and location that the image was captured, the type of CMOSimage sensor 311, mode setting information, integration timeinformation, taking lens unit setting information that characterizes theprocess used to capture the image, and processes, methods and algorithmsused by multimedia presentation device 301 to form the image. Themetadata can also include, but is not limited to, any other informationdetermined by digital processor 320 or stored in any memory inmultimedia presentation device 301 such as information that identifiesmultimedia presentation device 301, and/or instructions for rendering orotherwise processing the digital image with which the metadata isassociated. The metadata can also comprise an instruction to incorporatea particular message into digital image when presented. Such a messagecan be a text message to be rendered when the digital image is presentedor rendered. The metadata can also include audio signals. The metadatacan further include digital image data. The metadata can also includeany other information entered into multimedia presentation device 301.

The digital images and optional metadata, can be stored in a compressedform. For example, where the digital image comprises a sequence of stillimages, the still images can be stored in a compressed form such as byusing the JPEG (Joint Photographic Experts Group) ISO 10918-1 (ITU-T.81)standard. This JPEG compressed image data is stored using the so-called“Exif” image format defined in the Exchangeable Image File Formatversion 2.2 published by the Japan Electronics and InformationTechnology Industries Association JEITA CP-3451. Other image compressionand storage formats can be used. This format includes an Exifapplication segment that stores particular image metadata using variousTIFF tags. Separate TIFF tags can be used, for example, to store thedate and time the picture was captured, the lens f/number and othercamera settings, and to store image captions. In particular, the ImageDescription tag can be used to store labels. Real-time clock 324provides a capture date/time value, which is stored as date/timemetadata in each Exif image file.

Similarly, other compression systems including but not limited to theMPEG-4 (Motion Pictures Export Group) or Apple Quicktime™ standard canbe used to store digital images and metadata.

The processed digital images are stored in the image/data memory 330.Image/data memory 330 can also be used to store the personal profileinformation in person profile database 235. Image/data memory 330 canalso store other types of data, such as phone numbers, to-do lists, andthe like.

In the embodiment of FIG. 1, location determiner 325 provides thegeographic location associated with an image capture so that digitalprocessor 320 can incorporate such information into the metadata.Location determiner 325 can use any of a number of methods fordetermining the location of the image. For example, the geographiclocation can be determined by receiving communications from thewell-known Global Positioning Satellites (GPS). The location ispreferably stored in units of latitude and longitude, however, otherforms of location information can be used. Note that location determiner325 can determine the geographic location at a time slightly differentthan the image capture time. In that case, location determiner 325 canuse a geographic location from the nearest time as the geographiclocation associated with the image. Alternatively, location determiner325 can interpolate between multiple geographic positions at timesbefore and/or after the image capture time to determine the geographiclocation associated with the image capture. Interpolation can benecessitated because it is not always possible for location determiner325 to determine a geographic location. For example, the GPS receiversoften fail to detect signal when indoors. In this case, the lastsuccessful geographic location reading (i.e. prior to entering thebuilding) can be used by location determiner 325 to estimate thegeographic location associated with a particular image capture.

Digital processor 320 can also produce a low-resolution “thumbnail” sizeimage, which can be produced as described in commonly-assigned U.S. Pat.No. 5,164,831 to Kuchta et al., the disclosure of which is incorporatedby reference herein. The thumbnail image can be stored in RAM memory 322and supplied to a color display 332, which can be, for example, anactive matrix LCD or an organic light emitting diode (OLED) display.After images are captured, they can be quickly reviewed on the color LCDimage display 332 by using the thumbnail image data.

The graphical user interface displayed on color display 332 iscontrolled by user controls 334. Digital processor 320 is adapted toreceive user input and identified requests from user controls 334. Usercontrols 334 can include dedicated push buttons (e.g. a telephonekeypad) to dial a phone number, a control to set the mode (e.g. “phone”mode, “camera” mode), a joystick controller that includes 4-way control(up, down, left, right) and a push-button center “OK” switch, or thelike. Additional user controls 334 can comprise of a touch screenassociated with color display 332 such as found in the “KODAK EasyShareOne” digital camera or an attached mouse controller device associatedwith general control computer 375.

An audio codec 340 connected to the digital processor 320 receives anaudio signal from a microphone 342 and provides an audio signal to aspeaker 344. These components can be used both for telephoneconversations and to record and playback an audio track, along with avideo sequence or still image. Speaker 344 can also be used to informthe user of an incoming phone call. This can be done using a standardring tone stored in firmware memory 328, or by using a custom ring-tonedownloaded from a mobile phone network 358 and stored in image/datamemory 330. In addition, a vibration device (not shown) can be used toprovide a silent (e.g. non audible) notification of an incoming phonecall.

A dock interface 362 can be used to connect multimedia presentationdevice 301 to a dock/charger 364, which is connected to a generalcontrol computer 375. Dock interface 362 can conform to, for example,the well-know USB interface specification. Alternatively, the interfacebetween multimedia presentation device 301 and general control computer375 can be a wireless interface, such as the well-known Bluetoothwireless interface or the well-know 802.11b wireless interface. The dockinterface 362 can be used to download images from the image/data memory330 to the general control computer 375. The dock interface 362 can alsobe used to transfer calendar information from the general controlcomputer 375 to the image/data memory 330 in the multimedia presentationdevice 301. Dock/charger 364 can also be used to recharge the batteries(not shown) in digital multimedia presentation device 301.

In the embodiment of FIG. 1, digital processor 320 is coupled to anoptional wireless modem 350, which enables multimedia presentationdevice 301 to transmit and receive information via a Radio Frequency(RF) channel 352. A wireless modem 350 communicates over a radiofrequency (e.g. wireless) link with mobile phone network 358, such as a3GSM network. The mobile phone network 358 communicates with a photoservice provider 372 and companion database 327, which can exchangedigital images with multimedia presentation device 301. These images canalso be accessed via Internet 370 by other devices, including thegeneral control computer 375. Mobile phone network 358 also connects toa standard telephone network (not shown) in order to provide normaltelephone service. In addition, the radio frequency link can also accesspublic broadcast media such as television and radio from a remote mediasource 285. Broadcast media can also be accessed directly from a commoncable media provider such as Time Warner Cable, using a common cable TVinterface 307 or through mobile phone network 358 or through Internet370. Email, text or media messaging can be a remote media source 285 aswell.

One embodiment of a database and software block diagram is illustratedin FIG. 2. With brief reference back to FIG. 1, image/data memory 330,firmware memory 328, RAM memory 322 and digital processor 320 can beused to provide the necessary data storage and computational functionsas described below. The term “digital processor” is intended to be asynonym for or include any data processing device, such as a centralprocessing unit (“CPU”), a desktop computer, a laptop computer, amainframe computer, a personal digital assistant, a Blackberry™, or anyother device for processing data, managing data, or handling data,whether implemented with electrical, magnetic, optical, biologicalcomponents, or otherwise.

As shown in FIG. 2, an image database 200 is provided consisting of adigital image collection 210 and companion capture metadata 215 andimage content metadata 220. Remote media source 285 represents channelsor sources of visual media. Examples of visual media from a remote mediasource 285 include streaming media from television stations, a playingsong on radio stations, or Internet-based streaming content. Remotedatabase 245 can be personal libraries of media, media available fordownload and purchase or libraries of shared media collections fromother users or third parties. Remote database 245 can be accessed fromInternet 370 or the mobile phone network 358 or a photo service provider372. Person profile database 235 contains user information such as phonenumber, birthday, name, appearance information, addresses, likes,dislikes, and other information about a user. Additional personalprofile database methods are described in commonly assigned U.S. Pat.No. 5,652,880 to Seagraves, the disclosure of which is hereinincorporated by reference. Relational database 240 works in conjunctionwith person profile database 235 to provide tree structure orunstructured data storage known as “triples” used by artificialintelligence engines. Such data structures can provide data linkage toother people which can include friends, family, social networks, groupsor teams one belongs to. Linkages are also made using content-based textretrieval from media content. This can include objects, places andevents that are found in conjunction with people in media files usingimage content engine 280. Image content engine 280 is one of severalsoftware engines 290. In this embodiment, image content engine 280derives text-based descriptions of media using methods described incommonly assigned U.S. Pat. No. 5,697,001 to Ring et al., whichdescribes methods of extracting object data within images and isincorporated by reference herein. Briefly stated, a photographic cameraincludes an electronic image processing system comprising a computer andthe electronic camera, wherein the computer interfaces with the outputsection in the camera and accesses the image data and the object data inorder to characterize the objects within the image.

Event engine 225 clusters images into events producing event database258. Events may be a birthday party, vacation, collection of familymoments or a soccer game. Such events may also be broken intosub-events. A birthday party may comprise cake, presents, and outdooractivities. A vacation may be a series of sub-events associated withvarious cities, times of the day, visits to the beach etc. Events can betagged manually or can be clustered automatically. U.S. Pat. No.6,606,411, assigned to Loui et al., entitled “A method for automaticallyclassifying images into events,” issued Aug. 12, 2003 and, U.S. Pat. No.6,351,556, assigned to Loui et al., entitled “A method for automaticallycomparing content of images for classification into events”, issued Feb.26, 2002, disclose algorithms for clustering image content by temporalevents and sub-events. The disclosure of the above two patents areherein incorporated by reference. According to U.S. Pat. No. 6,606,411,events have proximate capture time & date and usually consistent colordistributions. Therefore, these pictures are likely to have been takenwith the same backdrop. For each sub-event, a single color and texturerepresentation is computed for all background areas taken together. Theabove two patents teach how to cluster images and videos in a digitalimage collection into temporal events and sub-events. The terms “event”and “sub-event” are used in an objective sense to indicate the productsof a computer mediated procedure that attempts to match a user'ssubjective perceptions of specific occurrences (corresponding to events)and divisions of those occurrences (corresponding to sub-events).Briefly summarized, a collection of images is classified into one ormore events determining one or more largest time differences of thecollection of images based on time and/or date clustering of the imagesand separating the plurality of images into the events based on havingone or more boundaries between events which one or more boundariescorrespond to the one or more largest time differences. For each event,sub-events (if any) can be determined by comparing the color histograminformation of successive images as described in U.S. Pat. No.6,351,556. Dividing an image into a number of blocks and then computingthe color histogram for each of the blocks can accomplish this. Ablock-based histogram correlation procedure is used as described in U.S.Pat. No. 6,351,556 to detect sub-event boundaries. Another method ofautomatically organizing images into events is disclosed in U.S. Pat.No. 6,915,011, assigned to Loui et al., entitled “Event clustering ofimages using foreground and background segmentation” issued Jul. 5,2005, which is herein incorporated by reference. Briefly summarized,according to one aspect of the above invention, an event clusteringmethod uses foreground and background segmentation for clustering imagesfrom a group into similar events. Initially, each image is divided intoa plurality of blocks, thereby providing block-based images. Utilizing ablock-by-block comparison, each block-based image is segmented into aplurality of regions comprising at least a foreground and a background.One or more luminosity, color, position or size features are extractedfrom the regions and the extracted features are utilized to estimate andcompare the similarity of the regions comprising the foreground andbackground in successive images in the group. Then, a measure of thetotal similarity between successive images is computed, therebyproviding image distance between successive images, and event clustersare delimited from the image distances.

Data mining engine 260 derives threads of semantic information throughanalysis of image database 200, remote database 245, event database 258,person profile database 235, and relational database 240. Thematicengine 270 determines themes from semantic information found by datamining engine 260. Rendering engine 265 produces the visual displayusing animation database 255, graphics database 250, and imagery derivedfrom image database 200, remote database 245, and remote media sources285.

Image value engine 275 is adapted to determine an image value for eachof the acquired images is determined by analyzing the images accordingto an image value metric. Methods that can be used to determine theimage value metric will be discussed in greater detail below.

FIG. 3 is a flow diagram showing a method of image-based presentation.Those skilled in the art will recognize that the display of image basedpresentations can be on the display of a cell phone, a displayassociated with a computing device, an imaging device, a digital mediaframe, a digital media player or a television or any other appliancepossessing a display for displaying images. Multiple displays can workin conjunction with these devices. In addition, multiple displays canpresent different presentations or the same presentation across thedevices.

At least one set of images is acquired for presentation. (Step 100).Images can be selected from digital image collection 210 from image/datamemory 330 or database 114. Images can be acquired with system-selectedrules. Such rules can be associated with most recent images downloaded,image metadata or a theme that can be system-produced or chosen by auser. Images can be a still image, a graphic, a video sequence or animage visualization of information, text or a data file.

A presentation area containing presentation objects is generated havinga presentation area image for presentation on color display 332 having aplurality of separate presentation objects in presentation areas. Asillustrated in FIG. 4, presentation area 405 represents a color display332 of multimedia presentation device 301. In some embodiments, colordisplay 332 can be small as a one-inch cell phone screen. In oneembodiment, multimedia presentation device 301 is a digital media playeror digital media wall frame. In another embodiment, a computer monitor,and in yet another embodiment multimedia presentation device 301 is adigital television set. Background 425 can be a non-animation areawithin color display 332 that at least two displayable object types suchas 410, 412, 415, 420 and 400 can move across. In one embodiment,background 425 can be a solid color. In other embodiments, background425 can be artwork associated with a theme from graphics database 250 orfrom an image or a moving video clip from image database 200 or remotedatabase 245. Large horizontal object 410 and large vertical object 412represent locations of images for image presentation. These largerdisplay areas are used for a set of images representing high value tothe user and high correlation to a selected theme. Mid-size presentationobjects 400 and 415 are for a set of content that is secondary to thatplaced in large horizontal object 410 and large vertical object 412.Small object 420 can be reserved for content that is ancillary to thetheme or not of high image value.

In FIGS. 5A-5C, what is shown is a representation of layers of images.Layers are one embodiment for selecting a stratification scheme forpresenting, at one time, the plurality of presentation objects in anordered manner within presentation area 405. The strata are defined bythe desired presentation attention for each ones of the ordered layers.Each respective layer can be associated with a range of image values,sets of acquired images or theme. Image objects are further providedwith a motion vector 445. As illustrated, motion vectors 445 canrepresent an instance of the motion of associated with a movement pathassociated with a logical mathematical function in the display area.Moreover, layer 430 is a representation of a collection of largepresentation objects and layer 435 is a representation of midsizepresentation objects and layer 440 is a representation of small objects420. In one embodiment, the cluster of objects in layer 430 will animatein a manner that is slower, appear in the foreground, and have a minimaltranslucency extent compared to presentation objects appearing in onlayer 435 and layer 440. It should be noted that three layers arediscussed in this example, however this is not limited and any number oflayer greater than one can be used. It will be understood that anynumber of layers appropriate to the display of multimedia presentationdevice 301 can be implemented within the scope of the invention.

An image value for each of the acquired images is determined byanalyzing the images according to an image value metric (Step 104). Animage value metric can comprise several components that will sum into asingle image value score. One component of image value can compriseimage quality. Another component can comprise the colorfulness of theimage. A further component can be that the eyes of a media subject areopen or that the subject is smiling. Another image value component canbe that the image media is aesthetically pleasing using rules of goodcomposition. Other components of image value are determined based inpart by automatically sensing who is watching or using multimediapresentation device 301. A user can be sensed based on recognition of auser based on a log-in ID, the use of a personal multimedia presentationdevice 301, such as the use of a personal cellular phone type ofmultimedia presentation device 301 by an owner or frequent user of thatcellular phone, or based on facial recognition using an embodiment thatincludes an imaging system in multimedia presentation device 301 tocapture an image of one or more users. From the image of the users, amood of the users can further be determined to create themes. Activitycan be determined to be a high-energy party, versus mellow and sittingon a couch. Furthermore, a personal based image value can be related thebehavior of one or more users. This behavior can be frequency ofviewing, utilizing certain media such as images tagged by the user orusers as favorites, or a frequency of viewing a media object from alibrary or remote content such as a television station or show. Othermethods of ranking personal based image value can be calculated inconjunction with how closely the person appearing in a media collectionis in relation to the sensed user(s). Best friends or closest relativescan have higher values associated with a sensed user than friends offriends or distant relatives as documented in relational database 240.

The presentation area image is then presented with one of the acquiredimages in each of the presentation objects and with the presented imagesbeing selected such that there will be at least a range of differentimage values for the images presented at any one moment in time (Step106). Image sets can be assigned to displayable presentation objects inassociation with image values. A presentation object characteristic canbe continuously variable according to image value. Objectcharacteristics are further associated with size, closeness to thecenter of the screen, speed, order, and translucency.

The presentation objects are then animated within the presentation areaimage by moving the presentation objects relative to each other in amanner that attracts more attention to presentation objects that areused to present images having a higher image value than presentationobjects that are used to simultaneously present images having a lowerimage value (Step 108). As a result, various presentation objects willattract various levels of attention. Lower image value images can movefaster through the presentation area while higher image value images canbe larger and move slowly, commanding greater level of viewer attention.In certain embodiments, the animation of various presentation objectscan be organized accordingly to what will be referred to herein as amotion metaphor. Motion metaphors that have large objects moving slowlyin conjunction with fast moving small objects have a psychologicalbenefit of maintaining viewer interest in the presentation. One suchmotion metaphor can be a solar system motion metaphor with smallpresentation objects apparently orbiting larger presentation objects.

Additional embodiments of the invention can incorporate themes for imagepresentation. A first theme for image presentation can be chosen usingfrom a list of possible themes or thematic engine 270 can generate afirst theme and a second theme using data mining engine 260. Thematicengine 270 can generate the first theme associated with a date, event orseason of images. In one embodiment, data mining engine 260 can query aperson profile database 235 to determine dates that are significant to auser. Such dates can include birthdays, anniversaries, faith-basedholidays, or national holidays. In this embodiment, an anniversary of anupcoming event may be selected as a suitable theme. If the earliestsignificant anniversary of a significant event within person profiledatabase 235 is December 25, and there are images associated with thatdate within the digital image collection 210, then on a viewing date ofDecember 21st, the theme can be of last year's Christmas or acompilation of previous family Christmas celebrations. In addition,first, second and subsequent themes can be chosen based on any singlemetadata or semantic elements within a person's picture collection. If arecurring subject of a person's picture collection is swimming, a themecan be selected as swimming. Another may be the most photographed personor object. One or a group of friends, family or social network canrepresent a theme for imagery to present using relational database 240.Furthermore, visualization of media types can be themes. A theme can beof all of a type of television stations that are available to the vieweror of the recorded media on a media player. A theme can be avisualization of unread email in an email or message computerapplication. A theme can be the most recently viewed pictures, or theleast recently viewed pictures in an image collection. Themes can beassociated with frequency of other viewer activity such as mostfrequented U-Tube videos watched or other ranking criteria associatedwith a ranking list. Theme lists can be presented in pull down menus ormanaged by a viewer or user. Themes can be system generated and selectedat random or selected with user controls 334 using associated key wordsfrom the images as capture metadata 215 and image content metadata 220.Event database 258 enables the data associated with an event to enablethe acquisition of images supporting a theme. For example, a group ofimages from an event can be classified as images that depict familymoments and/or family members. When a family theme is later determinedfor presentation, digital processor 320 can search any availabledatabase or image/data memory 330 for images that depict family members.Thus, the images from that event are suitable imagery for this theme.

The image value can be further modified in accordance with how each ofthe acquired images corresponds to a theme. Degree of correspondence toa theme can be related to the key words associated with an image incapture metadata 215, image content metadata 220 generated by imagecontent engine 280 by data mining engine 260. If the theme is camping,key words can include tent, hike, backpack, campfire, and outdoors.Through analyzing each key word, an image can have a degree ofcorrespondence to the theme of camping. In this example, an imagedepicting a tent and campfire can have a relatively high correspondenceto the camping theme while an image depicting people hiking or wearingbackpacks can have a lower correspondence to the camping theme.Furthermore, if a theme is camping, animation can be assigned based onthe theme of camping.

One example is illustrated in FIG. 6A, which shows a representation of atheme of presentation objects in a motion metaphor of panning alongwater and rocks in a stream. Higher valued images can be assigned to thelarge horizontal objects 410 that appear as rocks in an image. Lowerimage value images can be assigned to a “water” animation of smallobjects 420. As a result, the concurrent animation of panning ortranslating motion of a background image 425 or objects 410, 420provides an illusion of a larger presentation area 405.

FIG. 6B shows a metaphor of traveling that can be appropriate, forexample, when presenting images according to a navigating theme. Herelarge presentation object 527 can be centered and moves in a manner togrow in size on presentation area 405, as smaller presentation objectstranslated outward to the periphery of presentation area 405. Navigationtheme can be applied to an event of a vacation or road trip. Each set ofimages can be associated with a sub-event to the main event. Image fromthe sets depicting sub-event sets can be displayed on the presentationobjects concurrently or sequentially. Additionally, sets of images thatcan be defined by single or unique metadata elements thus enabling eachset to have unique combinations of images and/or unique presentationcharacteristics.

FIG. 6C shows another motion metaphor having a centered presentationobject and orbiting presentation objects. Motion metaphors can support atheme that centers on one set of images or preferred image. The imagehaving the highest image value is promoted as large horizontal object410 to attract the greatest attention while mid-sized presentationobject 400 and small object 420 are actively promoted in circular orelliptical orbits about large horizontal object 410 that are animated inaccordance with their image values. In this embodiment, equations ofmotion can dictate and be applied based on a mass related to the area ofthe object and related objects. In one example embodiment, a rate ofcircling can be determined using a calculated velocity that is based ona standard orbit equation.

FIG. 6D shows another motion metaphor for use with a connecting theme.In this motion metaphor, presentation objects are clustered in a mannerthat corresponds to the passage of time or the growth of someone orsomething over time. This animation starts with grouping presentationobject 560 and grows and produces connected images with each attachedpresentation object 565. This group of objects can move slowly acrossthe screen in a manner that appears that a camera is panning across ascene while additional grouping presentation object 570 speed and linkto the built up cluster of objects. These built up connected images ofpresentation objects 570 can be constructed using basic tessellationbuilding blocks in sub-clusters of image shapes. Presentation objectsare not limited to a 2:3 or 4:5 aspect ratio. Embodiments can includegroupings of cropped images in various sized and shaped presentationobjects. Commonly assigned U.S. Pat. No. 6,654,506 to Luo et al. andU.S. Pat. No. 6,654,507 to Luo, describe methods for automaticallycreating cropped and zoomed versions of photographic images wherein themethod and computer program/system for cropping a digital image includesselecting a zoom factor and a crop window, positioning the crop windowsuch that the crop window is centered on a main portion having a highestbelief value of being the main subject, and cropping the image accordingto the crop window and automatically producing an image of a portion ofa photographic image respectively. In addition, images can be croppedand divided into multiple image portions for presentation using variousaspect ratios. Commonly assigned U.S. Pat. No. 6,654,507 to Luodescribes a method and computer program/system for cropping a digitalimage includes selecting a zoom factor and a crop window, positioningthe crop window such that the crop window is centered on a main portionhaving a highest belief value of being the main subject, and croppingthe image according to the crop window. This main subject can beextracted using one crop window, and secondary subject can be extractedwith another crop window in sequence.

FIG. 7A shows a motion metaphor wherein presentation objects have shapesthat are related to the theme to provide visual theme context. In thisembodiment, the shape and border area of presentation object are alignedwith the shapes that are associated with a Christmas theme. Images arecropped to within the shape of the presentation objects. Accordingly,presentation objects are in the shape of snowflakes and Christmas trees.The arrangement at the beginning and throughout the presentation ofthese presentation objects enables a mix of object sizes and patterns.For an example of Christmas as an upcoming event, there can be manythemes associated with Christmas. A theme of “Christmas tree” is oneembodiment of a theme associated with Christmas. People in front of thetree are Theme One. Data mining engine 260 can filter images from a setof images captured in the month of December for the presence of thecolor green. If there is a large presence of people in front of a greenbackground that is close to the color of the tree, the degree ofcorrespondence for theme one is high. In addition, if people andChristmas tree ornaments are sensed, there is an even higher likelihoodthat there is a tree in the image and as a result, a high correspondenceto a Christmas tree set of images. Theme Two, getting the tree, can bedetermined by coats that the people are wearing, an outdoorclassification based on lighting or background, the presence of greenwithout ornaments, and snow in the picture. Referring back to FIG. 7A,Theme One (images of people in front of a tree), can be assigned to treepresentation object 535. Theme Two images can be assigned to snowflakepresentation object 530. Animation can comprise of snow falling motionvector 445 for snowflake presentation object 530 which occursconcurrently with an animation of a panning motion of tree presentationobject 535 suggesting movement up a hill of Christmas trees.

FIG. 7B shows a motion metaphor wherein presentation objects 540 haveshapes, such as balloon shapes, that are associated with a birthdayparty or celebration theme. Images from the birthday party can beinserted, zoomed and cropped within presentation objects 540 that have,for example, a balloon shape. In addition, animation of presentationobjects 540 can be logically determined from the shape of the balloonpresentation object 540 to provide an expected association fromknowledge of motion characteristics of a tangible object represented bythe shape of the presentation objects and/or determined based upon thetheme for the movement of the object. The type of animation associatedwith, for example, helium balloons is typically traveling from low tohigh such that motion vector 445 is pointed up. Confetti presentationobject 545, associated with a companion theme in this embodiment thattakes on a motion vector 445 influenced by simulated gravity, a randomBrownian motion, or swirling wind. With a swirling animation, objectposition, relative velocity and acceleration relationship can be modeledusing fluid equations. In addition, salient contextual images or imageobjects of the birthday party event group of images can be concurrentlydisplayed. With the birthday theme, a salient set of images include thebirthday child's baby pictures and/or images depicting the child atdifferent ages.

FIG. 7C shows motion metaphor wherein presentation objects 550 haveavian shapes and are moved in accordance with a flocking theme having aleader bird presentation object 550 and follower bird presentationobject 555 that can take on commonly known animations associated withflocking. Groups of bird shaped presentation objects 550 can flock in orout and swirl in and out of presentation area 405. Images can be croppedto the outline of the flocking bird presentation objects 550 and 555.Flocking birds can be selected for group activity events and themes.FIG. 7D shows a motion metaphor useful in representing a passage theme,and illustrates translating cloud presentation object 575 in many sizes.Images are cropped to the outline of cloud presentation objects 575 andtransverse across presentation area 405. Companion theme airplanepresentation object 580 can convey images of the person on the passagethrough space or time.

As with all of these embodiments, music and/or narration can play duringthe image presentation. Narration can be recorded and subsequentlyplayed during future presentations of the theme and images. Music can beassociated with the theme of the presentation such as Christmas or abirthday. Music can be associated with the era of a sequence of picturesfrom a certain year like a popular musical hit from that year. Music canaccompany the audio of a video presentation object. When a videopresentation object is encountered and subsequently presented, musicthat is accompanying the presentation is attenuated, not terminated, toallow the audio of the video presentation object to be heard while theattenuated music provides continuity in the overall presentation. Thepresentation including the accompanying music and video object withaudio can be further rendered to a format such as a DVD movie, slideshow or the like. Attenuation of accompanying music to accommodate avideo object is highly desirable. In other embodiments where multiplevideo clips are shown, the dominant video clip can have the loudestvolume over smaller also playing video clips. The video and audio inconjunction with a display object can be played or restarted with a userinteraction. With the user interaction, all other objects can mute. Insome embodiments, the motion vector 445 can be sized for the speed ofthe presentation object such that it transverses the screen for an exactamount of time. This time can be associated with the entire playbacktime of a whole or trimmed video snippet.

Referring to FIG. 8A, what is shown is a representation of media playingfrom a remote media source. As shown, media source outline 512 canappear over non-animated background 513 of a current channel. Mediasource outline 512 can represent current playing television channelsover the air or via cable TV or streaming over the Internet 370. Sizesof media source outline can represent favorite shows or stations thatsensed users prefer. Similarly, FIG. 8B represents media cover artwork515 from a library of a media collection. Artwork can be a record albumjacket, a CD jacket, a DVD or HD-DVD or similar media collection jacket.A playing selection can be represented as background 525 or as a movingor non-moving presentation object.

FIG. 9 is a representation of a menu of themes. A user can interact orclick on the presentation objects that represent the theme and canlaunch into the theme and subsequent images within objects presented.For example, year in pictures theme 510 can launch a presentationassociated with images captured in that year. Message theme 505 can linkto a presentation of messages. Presentation object, “this day last yeartheme” 500 appears in conjunction with other themes such as Internetlocation theme 495. Theme icons 465 present themes that are availablefor viewing. Sharing message theme 470 can launch into another user'simage collection from remote database 245. Service representation 475and store representation 490 illustrate visualization of goods andservices available in a theme presentation.

FIG. 10A shows one embodiment of an interaction with moving presentationobjects. A user control arrow 450 can click on a presentation object 455containing an image. Pressing on the image directly using a touch screeninterface on color display 332 can also perform this. As shown in FIG.10B, clicking on the presentation object 455 brings the object to theforeground and pauses the motion of it and other objects. It can alsoenlarge it to fill some or all of the screen. Resuming image motion canbe accomplished by clicking on the presentation object 455 again.Alternatively, clicking twice or right mouse clicking on presentationobject 455 can present to the user a menu of options associated withpresentation object 455. One set of options can present a list of allobjects within or associated with that image. By selecting theseobjects, one can launch new themes, connect, search or send a message tothe people within the images, or purchase products shown or representedwithin the image.

FIG. 11 shows the use of segmented portions of images used forpresentation. Any image segment can be extracted from an image. Salientshapes of people such as heads and or bodies can be extracted from theimages and the salient shapes are used as shapes to form presentationobjects 527 and 528. This can also be performed for video objects on aframe-by-frame basis. Similarly, a border of various styles, colors andpatterns can be placed around each of the presentation objects to denotethe set associated with the objects. As cited earlier, relationaldatabase 240 works in conjunction with person profile database 235 toprovide tree structure or unstructured data storage known as “triples”used by artificial intelligence engines. In one embodiment, the themeincorporates a primary set of content and at least one companion set ofcontent defined according to the tree structure or “triple”. A child canbe the primary theme with one border color or pattern portion. Closeassociates of the child for example, the mother, father, and siblingscan also be organized as companion sets of content, each with their ownshape, frame or manner of animation.

Those skilled in the art will recognize that many variations can be madeto the description of the present invention without significantlydeviating from the scope of the present invention.

Parts List

-   100 step-   102 step-   104 step-   106 step-   108 step-   200 image database-   210 digital image collection-   215 capture metadata-   220 image content metadata-   225 event engine-   235 person profile database-   240 relational database-   245 remote database-   250 graphics database-   255 animation database-   258 event database-   260 data mining engine-   265 rendering engine-   270 thematic engine-   275 image value engine-   280 image content engine-   285 remote media source-   290 software engines-   301 multimedia presentation device-   303 flash-   305 lens system-   307 cable TV interface-   311 CMOS image sensor-   312 timing generator-   314 image sensor array-   316 A/D converter circuit-   318 DRAM buffer memory-   320 digital processor-   322 RAM memory-   324 real-time clock-   325 location determiner-   327 database-   328 firmware memory-   330 image/data memory-   332 color display-   334 user controls-   340 audio codec-   342 microphone-   344 speaker-   350 wireless modem-   352 RF channel-   358 phone network-   362 dock interface-   364 dock/charger-   370 Internet-   372 service provider-   375 general control computer-   400 mid-size object-   405 presentation area-   410 large horizontal object-   412 large vertical object-   415 presentation object-   420 small object-   425 background image-   430 layer 1-   435 layer 2-   440 layer 3-   445 motion vector-   450 control arrow-   455 presentation object-   460 media source-   465 theme icon-   470 message theme-   475 service representation-   490 store representation-   495 Internet location-   500 this day last year theme-   505 message theme-   510 year in pictures theme-   512 media source outline-   513 background-   515 media library outline-   525 background-   527 large presentation object-   528 small presentation object-   530 snowflake presentation object-   535 tree presentation object-   540 balloon presentation object-   545 confetti presentation object-   550 leader bird presentation object-   555 follower bird presentation object-   560 grouping presentation object-   565 grouping presentation object-   570 grouping presentation object-   575 cloud presentation object-   580 airplane presentation object

The invention claimed is:
 1. A method for image presentation comprisingthe steps of: acquiring at least one set of images for presentation;generating a presentation area image for presentation on a displayhaving a plurality of separate presentation objects in the presentationarea; determining an image value for each of the acquired images byanalyzing the images according to an image value metric; presenting thepresentation area image with one of the acquired images in each of thepresentation objects with the presented images being selected such thatthere will be at least a range of different image values for the imagespresented at any one moment in time; animating the presentation objectswithin the presentation area image by moving the presentation objectsrelative to each other in a manner that attracts more attention topresentation objects that are used to present images having a higherimage value than presentation objects that are used to simultaneouslypresent images having a lower image value; and extracting salient shapesfrom the images and wherein the salient shapes are used as shapes toform the presentation objects for the presentation images that are usedto present portions of the images.
 2. The method of claim 1, wherein thepresentation objects are animated to attract more attention by providingvariations in at least one of the following: a velocity of movement ofthe presentation objects, an acceleration of movement of thepresentation objects, or a pattern of movement of the presentationobjects.
 3. The method of claim 1, further comprising determining atheme wherein the at least one set of images is acquired based upon thetheme.
 4. The method of claim 3, wherein the image value for each of theacquired images is further determined by a degree of correspondence tothe theme.
 5. The method of claim 1, wherein the presentation objectsare further actively promoted to attract more attention relative to eachother by providing variations in at least one of the following: a sizeof displayable objects, a shape of the displayable objects, anarrangement of the displayable objects before animation, an arrangementof the displayable objects after animation, the appearance of a borderportion of the displayable objects, or an extent of a translucency ofthe displayable objects.
 6. The method of claim 1, wherein the imageassociated with the highest image value comprises a motion video imagehaving a source audio track or sequences of images associated with asource audio track, wherein a primary audio track is provided duringpresentation of the images, and wherein the primary audio track is basedupon the source audio track.
 7. The method of claim 1, wherein theduration of presentation of a presentation object on the screen is basedon the length of the video segment to enable entire playback of thevideo segment.
 8. The method of claim 1, wherein the manner of animationof two of presentation objects is calculated such that they appear tomove relative to each other in a manner according to a relationshipdictated by a logical or mathematical function.
 9. The method of claim1, wherein the size of the displayable object is determined by itsmanner of animation whereby a larger object moves slower than a smallerobject.
 10. The method of claim 1, wherein the animation occursconcurrently with a panning or translating motion of a background imageto provide an illusion of a larger presentation space.
 11. The method ofclaim 1, wherein a user interaction on a presentation object thatrepresents a set of images launches the represented set into ananimation sequence.
 12. The method of claim 3, wherein the presentationobjects have shapes that are related to the theme so as to visuallysuggest a context based upon the theme.
 13. The method of claim 3,wherein the manner of animation is related to the theme to provide ananimation consistent with the theme.
 14. The method of claim 3, whereina primary audio track is provided during presentation of the images, andwherein said primary audio track comprises music selected based upon thetheme to provide contextual audio information.
 15. The method of claim3, where a theme is determined by analysis of the organization of aportion of a tree structure of a relational database used to store imagecontent information for the images used for presentation wherein thetheme incorporates a primary set of images and at least one companionset of images defined according to the tree structure.
 16. The method ofclaim 3, where the theme represents a grouping of images associated withan event and the acquired sets of images for presentation are inassociation with the event theme; further comprising image sets from thesub-events that make up the event.
 17. The method of claim 3, wherein atheme of presentation comprises an acquired set of images organized withreference to one person and wherein additional acquired sets areorganized according to persons that are known to be associates of theone person.
 18. The method of claim 17, further comprising acquired setsof images according to groups of individuals that reflect a socialnetwork associated with the primary person.
 19. A method for imagepresentation comprising the steps of: acquiring at least one set ofimages for presentation; generating a presentation area image forpresentation on a display having a plurality of separate presentationobjects in the presentation area; determining an image value for each ofthe acquired images by analyzing the images according to an image valuemetric; presenting the presentation area image with one of the acquiredimages in each of the presentation objects with the presented imagesbeing selected such that there will be at least a range of differentimage values for the images presented at any one moment in time;animating the presentation objects within the presentation area image bymoving the presentation objects relative to each other in a manner thatattracts more attention to presentation objects that are used to presentimages having a higher image value than presentation objects that areused to simultaneously present images having a lower image value;determining a theme wherein the at least one set of images is acquiredbased upon the theme, wherein the presentation objects have shapes thatare related to the theme so as to visually suggest a context based uponthe theme; and determining from the shape of the presentation object, aclear context for the movement of the object.
 20. A method for imagepresentation comprising the steps of: acquiring at least one set ofimages for presentation; generating a presentation area image forpresentation on a display having a plurality of separate presentationobjects in the presentation area; determining an image value for each ofthe acquired images by analyzing the images according to an image valuemetric; presenting the presentation area image with one of the acquiredimages in each of the presentation objects with the presented imagesbeing selected such that there will be at least a range of differentimage values for the images presented at any one moment in time;animating the presentation objects within the presentation area image bymoving the presentation objects relative to each other in a manner thatattracts more attention to presentation objects that are used to presentimages having a higher image value than presentation objects that areused to simultaneously present images having a lower image value;determining a theme wherein the at least one set of images is acquiredbased upon the theme; determining a background image based upon thetheme, wherein the presentation area image has the background image,and; automatically segmenting the background image into at least ananimation segment and a non-animation segment said animation segmentcontaining image information related to the theme and wherein and thepresentation objects are animated only within the animation segment.